Repository navigation
fix(system): honor scan.snapshot-id in metadata system tables - #498
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #497
FilesSystemTableandManifestsSystemTablealways read the latest snapshot, even when thecaller sets
scan.snapshot-id. A reader that pins a snapshot therefore gets file and manifestmetadata from a newer commit, and a snapshot id that does not exist silently falls back to the
latest snapshot.
This change replaces
LatestSnapshot()withSelectedSnapshot(), which readsCoreOptions::GetScanSnapshotId():Status::NotExist("snapshot <id> does not exist")instead of a fallback.ReadLatestManifestEntries/ReadLatestDataFilesare renamed toReadSelectedManifestEntries/ReadSelectedDataFilesto match.ManifestsSystemTablenowbuilds
CoreOptionsbefore selecting the snapshot.Other time travel options (
scan.tag-name,scan.timestamp-millis, ...) are out of scope.Tables that list history (
$snapshots,$schemas,$tags, ...) are unchanged.Tests
SystemTableReadInteTest.TestReadMetadataSystemTablesAtPinnedSnapshot: commits twosnapshots to a primary-key table, then checks that
$filesreturns both files;scan.snapshot-id = 1,$filesreturns only the first file (record_count = 1) and$manifestsreturns fewer manifests than the latest snapshot;scan.snapshot-id = 3, both tables fail withsnapshot 3 does not exist.Verified on Ubuntu 24.04 (aarch64), GCC 13, Debug:
paimon-read-inte-testpasses (303 tests),including the 31
SystemTableReadInteTestcases.API and Format
No public API change (only an internal file under
src/). No storage format or protocol change.Documentation
No.
Generative AI tooling
Generated-by: Claude Code (Claude Opus 5.5)